{"cells":[{"cell_type":"markdown","metadata":{"slideshow":{"slide_type":"slide"}},"source":["# Part 4 - What to enrich with? (What are Data Collections and Analysis Variables?)"]},{"cell_type":"markdown","metadata":{},"source":["## Data Collections and GeoEnrichment coverage\n","\n","As described earlier, a data collection is a preassembled list of attributes that will be used to enrich the input features. Collection attributes can describe various types of information, such as demographic characteristics and geographic context of the locations or areas submitted as input features. \n","\n","Some data collections (such as default) can be used in all supported countries. Other data collections may only be available in one or a collection of countries. [Data Browser](https://doc.arcgis.com/en/esri-demographics/data/data-browser.htm) can be used to examine the entire global listing of variables, and associated datasets for each country."]},{"cell_type":"markdown","metadata":{},"source":["<div class=\"embed-container\"><iframe frameborder=\"false\" scrolling=\"no\" width=\"800\" height=\"600\" marginheight=\"0\" marginwidth=\"0\" \n","src=\"https://la.arcgis.com/databrowser/index.html\"></iframe></div>"]},{"cell_type":"markdown","metadata":{},"source":["### List Countries with GeoEnrichment Data"]},{"cell_type":"markdown","metadata":{"slideshow":{"slide_type":"subslide"}},"source":["The `get_countries()` method can be used to query the countries for which GeoEnrichment data is available, and it returns a list of `Country` objects with which you can further query for properties. This list can also be viewed [here](https://developers.arcgis.com/rest/geoenrichment/api-reference/geoenrichment-coverage.htm)."]},{"cell_type":"code","execution_count":1,"metadata":{},"outputs":[],"source":["from arcgis.gis import GIS\n","from arcgis.geoenrichment import Country, enrich, get_countries"]},{"cell_type":"code","execution_count":2,"metadata":{},"outputs":[],"source":["# Create a GIS Connection\n","gis = GIS(profile='your_online_profile')"]},{"cell_type":"code","execution_count":3,"metadata":{"slideshow":{"slide_type":"fragment"}},"outputs":[{"name":"stdout","output_type":"stream","text":["Number of countries for which GeoEnrichment data is available: 177\n"]},{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>iso2</th>\n","      <th>iso3</th>\n","      <th>name</th>\n","      <th>alt_name</th>\n","      <th>datasets</th>\n","      <th>default_dataset</th>\n","      <th>continent</th>\n","      <th>hierarchy</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>AL</td>\n","      <td>ALB</td>\n","      <td>Albania</td>\n","      <td>ALBANIA</td>\n","      <td>[ALB_MBR_2021]</td>\n","      <td>ALB_MBR_2021</td>\n","      <td>Europe</td>\n","      <td>[census]</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>DZ</td>\n","      <td>DZA</td>\n","      <td>Algeria</td>\n","      <td>ALGERIA</td>\n","      <td>[DZA_MBR_2021]</td>\n","      <td>DZA_MBR_2021</td>\n","      <td>Africa</td>\n","      <td>[census]</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>AD</td>\n","      <td>AND</td>\n","      <td>Andorra</td>\n","      <td>ANDORRA</td>\n","      <td>[AND_MBR_2021]</td>\n","      <td>AND_MBR_2021</td>\n","      <td>Europe</td>\n","      <td>[census]</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>AO</td>\n","      <td>AGO</td>\n","      <td>Angola</td>\n","      <td>ANGOLA</td>\n","      <td>[AGO_MBR_2021]</td>\n","      <td>AGO_MBR_2021</td>\n","      <td>Africa</td>\n","      <td>[census]</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>AI</td>\n","      <td>AIA</td>\n","      <td>Anguilla</td>\n","      <td>ANGUILLA</td>\n","      <td>[AIA_MBR_2020]</td>\n","      <td>AIA_MBR_2020</td>\n","      <td>North America</td>\n","      <td>[census]</td>\n","    </tr>\n","    <tr>\n","      <th>5</th>\n","      <td>AR</td>\n","      <td>ARG</td>\n","      <td>Argentina</td>\n","      <td>ARGENTINA</td>\n","      <td>[ARG_MBR_2020]</td>\n","      <td>ARG_MBR_2020</td>\n","      <td>South America</td>\n","      <td>[census]</td>\n","    </tr>\n","    <tr>\n","      <th>6</th>\n","      <td>AM</td>\n","      <td>ARM</td>\n","      <td>Armenia</td>\n","      <td>ARMENIA</td>\n","      <td>[ARM_MBR_2020]</td>\n","      <td>ARM_MBR_2020</td>\n","      <td>Europe</td>\n","      <td>[census]</td>\n","    </tr>\n","    <tr>\n","      <th>7</th>\n","      <td>AW</td>\n","      <td>ABW</td>\n","      <td>Aruba</td>\n","      <td>ARUBA</td>\n","      <td>[ABW_MBR_2020]</td>\n","      <td>ABW_MBR_2020</td>\n","      <td>North America</td>\n","      <td>[census]</td>\n","    </tr>\n","    <tr>\n","      <th>8</th>\n","      <td>AU</td>\n","      <td>AUS</td>\n","      <td>Australia</td>\n","      <td>AUSTRALIA</td>\n","      <td>[AUS_ABS_2016, AUS_MBR_2020]</td>\n","      <td>AUS_ABS_2016</td>\n","      <td>Oceania</td>\n","      <td>[AUS_ABS, census]</td>\n","    </tr>\n","    <tr>\n","      <th>9</th>\n","      <td>AT</td>\n","      <td>AUT</td>\n","      <td>Austria</td>\n","      <td>AUSTRIA</td>\n","      <td>[AUT_MBR_2021]</td>\n","      <td>AUT_MBR_2021</td>\n","      <td>Europe</td>\n","      <td>[census]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["  iso2 iso3       name   alt_name                      datasets  \\\n","0   AL  ALB    Albania    ALBANIA                [ALB_MBR_2021]   \n","1   DZ  DZA    Algeria    ALGERIA                [DZA_MBR_2021]   \n","2   AD  AND    Andorra    ANDORRA                [AND_MBR_2021]   \n","3   AO  AGO     Angola     ANGOLA                [AGO_MBR_2021]   \n","4   AI  AIA   Anguilla   ANGUILLA                [AIA_MBR_2020]   \n","5   AR  ARG  Argentina  ARGENTINA                [ARG_MBR_2020]   \n","6   AM  ARM    Armenia    ARMENIA                [ARM_MBR_2020]   \n","7   AW  ABW      Aruba      ARUBA                [ABW_MBR_2020]   \n","8   AU  AUS  Australia  AUSTRALIA  [AUS_ABS_2016, AUS_MBR_2020]   \n","9   AT  AUT    Austria    AUSTRIA                [AUT_MBR_2021]   \n","\n","  default_dataset      continent          hierarchy  \n","0    ALB_MBR_2021         Europe           [census]  \n","1    DZA_MBR_2021         Africa           [census]  \n","2    AND_MBR_2021         Europe           [census]  \n","3    AGO_MBR_2021         Africa           [census]  \n","4    AIA_MBR_2020  North America           [census]  \n","5    ARG_MBR_2020  South America           [census]  \n","6    ARM_MBR_2020         Europe           [census]  \n","7    ABW_MBR_2020  North America           [census]  \n","8    AUS_ABS_2016        Oceania  [AUS_ABS, census]  \n","9    AUT_MBR_2021         Europe           [census]  "]},"execution_count":3,"metadata":{},"output_type":"execute_result"}],"source":["countries = get_countries()\n","print(\"Number of countries for which GeoEnrichment data is available: \" + str(len(countries)))\n","\n","#print a few countries for a sample\n","countries[0:10]"]},{"cell_type":"markdown","metadata":{},"source":["### Data Collections for U.S.\n","\n","The `data_collections` property of a `Country` object lists its available data collections and analysis variables under each data collection as a Pandas dataframe.\n","\n","In order to discover the data collections for a particular country, you may first access the reference variable to it using the `country.get()` method, and then fetch the data collections from `country.data_collections` property. Once we know the data collection we would like to use, we can look at `analysisVariable`s available in that data collection."]},{"cell_type":"code","execution_count":4,"metadata":{},"outputs":[{"data":{"text/plain":["arcgis.geoenrichment.enrichment.Country"]},"execution_count":4,"metadata":{},"output_type":"execute_result"}],"source":["# Get US as a country\n","usa = Country.get('US')\n","type(usa)"]},{"cell_type":"code","execution_count":5,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>analysisVariable</th>\n","      <th>alias</th>\n","      <th>fieldCategory</th>\n","      <th>vintage</th>\n","    </tr>\n","    <tr>\n","      <th>dataCollectionID</th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>1yearincrements</th>\n","      <td>1yearincrements.AGE0_CY</td>\n","      <td>2022 Population Age &lt;1</td>\n","      <td>2022 Age: 1 Year Increments (Esri)</td>\n","      <td>2022</td>\n","    </tr>\n","    <tr>\n","      <th>1yearincrements</th>\n","      <td>1yearincrements.AGE1_CY</td>\n","      <td>2022 Population Age 1</td>\n","      <td>2022 Age: 1 Year Increments (Esri)</td>\n","      <td>2022</td>\n","    </tr>\n","    <tr>\n","      <th>1yearincrements</th>\n","      <td>1yearincrements.AGE2_CY</td>\n","      <td>2022 Population Age 2</td>\n","      <td>2022 Age: 1 Year Increments (Esri)</td>\n","      <td>2022</td>\n","    </tr>\n","    <tr>\n","      <th>1yearincrements</th>\n","      <td>1yearincrements.AGE3_CY</td>\n","      <td>2022 Population Age 3</td>\n","      <td>2022 Age: 1 Year Increments (Esri)</td>\n","      <td>2022</td>\n","    </tr>\n","    <tr>\n","      <th>1yearincrements</th>\n","      <td>1yearincrements.AGE4_CY</td>\n","      <td>2022 Population Age 4</td>\n","      <td>2022 Age: 1 Year Increments (Esri)</td>\n","      <td>2022</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                         analysisVariable                   alias  \\\n","dataCollectionID                                                    \n","1yearincrements   1yearincrements.AGE0_CY  2022 Population Age <1   \n","1yearincrements   1yearincrements.AGE1_CY   2022 Population Age 1   \n","1yearincrements   1yearincrements.AGE2_CY   2022 Population Age 2   \n","1yearincrements   1yearincrements.AGE3_CY   2022 Population Age 3   \n","1yearincrements   1yearincrements.AGE4_CY   2022 Population Age 4   \n","\n","                                       fieldCategory vintage  \n","dataCollectionID                                              \n","1yearincrements   2022 Age: 1 Year Increments (Esri)    2022  \n","1yearincrements   2022 Age: 1 Year Increments (Esri)    2022  \n","1yearincrements   2022 Age: 1 Year Increments (Esri)    2022  \n","1yearincrements   2022 Age: 1 Year Increments (Esri)    2022  \n","1yearincrements   2022 Age: 1 Year Increments (Esri)    2022  "]},"execution_count":5,"metadata":{},"output_type":"execute_result"}],"source":["usa_df = usa.data_collections\n","\n","# print a few rows of the DataFrame\n","usa_df.head()"]},{"cell_type":"code","execution_count":6,"metadata":{},"outputs":[{"data":{"text/plain":["(18946, 4)"]},"execution_count":6,"metadata":{},"output_type":"execute_result"}],"source":["usa_df.shape"]},{"cell_type":"markdown","metadata":{},"source":["#### Unique Data Collections for U.S."]},{"cell_type":"markdown","metadata":{},"source":["Each data collection and analysis variable has a unique ID. When calling the `enrich()` method (explained earlier in this guide) these analysis variables can be passed in the `data_collections` and `analysis_variables` parameters.\n","\n","As an example, here we see a subset of the data collections for US showing 2 different data collections and multiple analysis variables for each collection."]},{"cell_type":"code","execution_count":7,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>analysisVariable</th>\n","      <th>alias</th>\n","      <th>fieldCategory</th>\n","      <th>vintage</th>\n","    </tr>\n","    <tr>\n","      <th>dataCollectionID</th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>1yearincrements</th>\n","      <td>1yearincrements.FAGE75_FY</td>\n","      <td>2027 Females Age 75</td>\n","      <td>2027 Age: 1 Year Increments (Esri)</td>\n","      <td>2027</td>\n","    </tr>\n","    <tr>\n","      <th>1yearincrements</th>\n","      <td>1yearincrements.FAGE76_FY</td>\n","      <td>2027 Females Age 76</td>\n","      <td>2027 Age: 1 Year Increments (Esri)</td>\n","      <td>2027</td>\n","    </tr>\n","    <tr>\n","      <th>1yearincrements</th>\n","      <td>1yearincrements.FAGE77_FY</td>\n","      <td>2027 Females Age 77</td>\n","      <td>2027 Age: 1 Year Increments (Esri)</td>\n","      <td>2027</td>\n","    </tr>\n","    <tr>\n","      <th>1yearincrements</th>\n","      <td>1yearincrements.FAGE78_FY</td>\n","      <td>2027 Females Age 78</td>\n","      <td>2027 Age: 1 Year Increments (Esri)</td>\n","      <td>2027</td>\n","    </tr>\n","    <tr>\n","      <th>1yearincrements</th>\n","      <td>1yearincrements.FAGE79_FY</td>\n","      <td>2027 Females Age 79</td>\n","      <td>2027 Age: 1 Year Increments (Esri)</td>\n","      <td>2027</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>5yearincrements</th>\n","      <td>5yearincrements.MEDAGE_CY</td>\n","      <td>2022 Median Age</td>\n","      <td>2022 Age: 5 Year Increments (Esri)</td>\n","      <td>2022</td>\n","    </tr>\n","    <tr>\n","      <th>5yearincrements</th>\n","      <td>5yearincrements.MALES_CY</td>\n","      <td>2022 Male Population</td>\n","      <td>2022 Age: 5 Year Increments (Esri)</td>\n","      <td>2022</td>\n","    </tr>\n","    <tr>\n","      <th>5yearincrements</th>\n","      <td>5yearincrements.MALE0_CY</td>\n","      <td>2022 Males Age 0-4</td>\n","      <td>2022 Age: 5 Year Increments (Esri)</td>\n","      <td>2022</td>\n","    </tr>\n","    <tr>\n","      <th>5yearincrements</th>\n","      <td>5yearincrements.MALE5_CY</td>\n","      <td>2022 Males Age 5-9</td>\n","      <td>2022 Age: 5 Year Increments (Esri)</td>\n","      <td>2022</td>\n","    </tr>\n","    <tr>\n","      <th>5yearincrements</th>\n","      <td>5yearincrements.MALE10_CY</td>\n","      <td>2022 Males Age 10-14</td>\n","      <td>2022 Age: 5 Year Increments (Esri)</td>\n","      <td>2022</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>100 rows × 4 columns</p>\n","</div>"],"text/plain":["                           analysisVariable                 alias  \\\n","dataCollectionID                                                    \n","1yearincrements   1yearincrements.FAGE75_FY   2027 Females Age 75   \n","1yearincrements   1yearincrements.FAGE76_FY   2027 Females Age 76   \n","1yearincrements   1yearincrements.FAGE77_FY   2027 Females Age 77   \n","1yearincrements   1yearincrements.FAGE78_FY   2027 Females Age 78   \n","1yearincrements   1yearincrements.FAGE79_FY   2027 Females Age 79   \n","...                                     ...                   ...   \n","5yearincrements   5yearincrements.MEDAGE_CY       2022 Median Age   \n","5yearincrements    5yearincrements.MALES_CY  2022 Male Population   \n","5yearincrements    5yearincrements.MALE0_CY    2022 Males Age 0-4   \n","5yearincrements    5yearincrements.MALE5_CY    2022 Males Age 5-9   \n","5yearincrements   5yearincrements.MALE10_CY  2022 Males Age 10-14   \n","\n","                                       fieldCategory vintage  \n","dataCollectionID                                              \n","1yearincrements   2027 Age: 1 Year Increments (Esri)    2027  \n","1yearincrements   2027 Age: 1 Year Increments (Esri)    2027  \n","1yearincrements   2027 Age: 1 Year Increments (Esri)    2027  \n","1yearincrements   2027 Age: 1 Year Increments (Esri)    2027  \n","1yearincrements   2027 Age: 1 Year Increments (Esri)    2027  \n","...                                              ...     ...  \n","5yearincrements   2022 Age: 5 Year Increments (Esri)    2022  \n","5yearincrements   2022 Age: 5 Year Increments (Esri)    2022  \n","5yearincrements   2022 Age: 5 Year Increments (Esri)    2022  \n","5yearincrements   2022 Age: 5 Year Increments (Esri)    2022  \n","5yearincrements   2022 Age: 5 Year Increments (Esri)    2022  \n","\n","[100 rows x 4 columns]"]},"execution_count":7,"metadata":{},"output_type":"execute_result"}],"source":["usa_df.iloc[500:600,:]"]},{"cell_type":"markdown","metadata":{},"source":["The table above shows 2 different data collections (1yearincrements and 5yearincrements). Since these are `Age` data collections, the `analysisVariable`s for these collections are similar. `vintage` shows the year that the demographic data represents. For example, a vintage of 2020 means that the data represents the year 2020.\n","\n","Let's get a list of unique data collections that are available for U.S. "]},{"cell_type":"code","execution_count":8,"metadata":{},"outputs":[{"data":{"text/plain":["115"]},"execution_count":8,"metadata":{},"output_type":"execute_result"}],"source":["usa_df.index.nunique()"]},{"cell_type":"markdown","metadata":{},"source":["*United States has 150 unique data collections.* Here are the first 10 data collections.\n"]},{"cell_type":"code","execution_count":9,"metadata":{},"outputs":[{"data":{"text/plain":["['1yearincrements',\n"," '5yearincrements',\n"," 'Age',\n"," 'agebyracebysex',\n"," 'AgeDependency',\n"," 'AtRisk',\n"," 'AutomobilesAutomotiveProducts',\n"," 'BabyProductsToysGames',\n"," 'basicFactsForMobileApps',\n"," 'businesses']"]},"execution_count":9,"metadata":{},"output_type":"execute_result"}],"source":["list(usa_df.index.unique())[:10]"]},{"cell_type":"markdown","metadata":{},"source":["Looking at `fieldCategory` is a great way to clearly understand what the data collection is about. `fieldCategory` combines vintage, datacollectionID columns along with the year and data collection. However, to query a data collection its unique ID (`dataCollectionID`) must be used.\n","\n","Let's look at the `fieldCategory` column for a few data collections in US."]},{"cell_type":"code","execution_count":10,"metadata":{},"outputs":[{"data":{"text/plain":["array(['2022 Age: 1 Year Increments (Esri)',\n","       '2027 Age: 1 Year Increments (Esri)',\n","       '2010 Age: 1 Year Increments (U.S. Census)',\n","       '2022 Age: 5 Year Increments (Esri)',\n","       '2027 Age: 5 Year Increments (Esri)',\n","       '2010 Age: 5 Year Increments (U.S. Census)',\n","       '2016-2020 Age: 5 Year Increments (ACS)',\n","       '2022 Age by Sex by Race (Esri)', '2027 Age by Sex by Race (Esri)',\n","       '2010 Age by Sex by Race (U.S. Census)'], dtype=object)"]},"execution_count":10,"metadata":{},"output_type":"execute_result"}],"source":["usa_df.fieldCategory.unique()[:10]"]},{"cell_type":"markdown","metadata":{},"source":["#### Data Collections by Socio-demographic Factors"]},{"cell_type":"markdown","metadata":{},"source":["You can filter the `data_collections` to get collections for a specific factor using Pandas expressions. Let's loook at data collections for different `socio-demographic factors` such as `Age, Population, Income`."]},{"cell_type":"markdown","metadata":{},"source":["__Data Collections for Age__"]},{"cell_type":"code","execution_count":11,"metadata":{},"outputs":[{"data":{"text/plain":["array(['2022 Age: 1 Year Increments (Esri)',\n","       '2027 Age: 1 Year Increments (Esri)',\n","       '2010 Age: 1 Year Increments (U.S. Census)',\n","       '2022 Age: 5 Year Increments (Esri)',\n","       '2027 Age: 5 Year Increments (Esri)',\n","       '2010 Age: 5 Year Increments (U.S. Census)',\n","       '2016-2020 Age: 5 Year Increments (ACS)',\n","       '2022 Age by Sex by Race (Esri)', '2027 Age by Sex by Race (Esri)',\n","       '2010 Age by Sex by Race (U.S. Census)',\n","       '2022 Age Dependency (Esri)', '2027 Age Dependency (Esri)',\n","       '2022 Disposable Income by Age (Esri)',\n","       '2010 Households by Age of Householder (U.S. Census)',\n","       '2016-2020 Households by Type and Size and Age (ACS)',\n","       '2010 Housing by Age of Householder (U.S. Census)',\n","       '2022 Income by Age (Esri)', '2027 Income by Age (Esri)',\n","       '2016-2020 Income by Age (ACS)', 'Age: 5 Year Increments',\n","       '2022 Net Worth by Age (Esri)',\n","       '2016-2020 Females by Age of Children and Employment Status (ACS)'],\n","      dtype=object)"]},"execution_count":11,"metadata":{},"output_type":"execute_result"}],"source":["Age_Collections = usa_df['fieldCategory'].str.contains('Age', na=False)\n","usa_df[Age_Collections].fieldCategory.unique()"]},{"cell_type":"markdown","metadata":{},"source":["__Data Collections for Population__"]},{"cell_type":"code","execution_count":12,"metadata":{},"outputs":[{"data":{"text/plain":["array(['2010 Population (U.S. Census)',\n","       '2016-2020 Population by Language Spoken at Home (ACS)',\n","       '2022 Daytime Population (Esri)',\n","       '2022 Population by Generation (Esri)',\n","       '2027 Population by Generation (Esri)',\n","       '2020 Group Quarters Population (U.S. Census)',\n","       '2020 Group Quarters Population by Type (U.S. Census)',\n","       '2010 Group Quarters Population (U.S. Census)',\n","       '2020 Hispanic Population by Race (U.S. Census)',\n","       '2020 Hispanic Population of Two or More Races (U.S. Census)',\n","       '2020 Hispanic Population <18 Years by Race (U.S. Census)',\n","       '2020 Hispanic Population 18+ Years by Race (U.S. Census)',\n","       '2020 Hispanic Population 18+ Years of Two or More Races (U.S. Census)',\n","       '2022 Population Time Series (Esri)',\n","       '2010 Population by Relationship and Household Type (U.S. Census)',\n","       '2016-2020 Population by Relationship and Household Type (ACS)',\n","       '2020 Non Hispanic Population by Race (U.S. Census)',\n","       '2020 Non Hispanic Population of Two or More Races (U.S. Census)',\n","       '2020 Non Hispanic Population <18 Years by Race (U.S. Census)',\n","       '2020 Non Hispanic Population 18+ Years by Race (U.S. Census)',\n","       '2020 Non Hispanic Population 18+ Years of Two or More Races (U.S. Census)',\n","       '2022 Population (Esri)', '2020 Population (U.S. Census)',\n","       '2020 Population by Race (U.S. Census)',\n","       '2020 Population of Two or More Races (U.S. Census)',\n","       '2020 Population <18 Years by Race (U.S. Census)',\n","       '2020 Population 18+ Years by Race (U.S. Census)',\n","       '2020 Population 18+ Years of Two or More Races (U.S. Census)'],\n","      dtype=object)"]},"execution_count":12,"metadata":{},"output_type":"execute_result"}],"source":["Pop_Collections = usa_df['fieldCategory'].str.contains('Population', na=False)\n","usa_df[Pop_Collections].fieldCategory.unique()"]},{"cell_type":"markdown","metadata":{},"source":["__Data Collections for Income__"]},{"cell_type":"code","execution_count":13,"metadata":{},"outputs":[{"data":{"text/plain":["Index(['1yearincrements', '5yearincrements', 'Age', 'agebyracebysex',\n","       'AgeDependency', 'AtRisk', 'AutomobilesAutomotiveProducts',\n","       'BabyProductsToysGames', 'basicFactsForMobileApps', 'businesses',\n","       ...\n","       'travelMPI', 'unitsinstructure', 'urbanizationgroupsNEW', 'vacant',\n","       'vehiclesavailable', 'veterans', 'Wealth', 'women', 'yearbuilt',\n","       'yearmovedin'],\n","      dtype='object', name='dataCollectionID', length=115)"]},"execution_count":13,"metadata":{},"output_type":"execute_result"}],"source":["Income_Collections = usa_df['fieldCategory'].str.contains('Income', na=False)\n","Income_Collections.index.unique()"]},{"cell_type":"markdown","metadata":{},"source":["As mentioned earlier, using a `data_collection`'s unique ID (`dataCollectionID`) is the best way to further query a data collection. Let's look at the `dataCollectionID` for various Income data collections."]},{"cell_type":"code","execution_count":14,"metadata":{},"outputs":[{"data":{"text/plain":["Index(['AtRisk', 'basicFactsForMobileApps', 'disposableincome',\n","       'foodstampsSNAP', 'Health', 'householdincome', 'households',\n","       'incomebyage', 'KeyUSFacts', 'Policy', 'population', 'Wealth'],\n","      dtype='object', name='dataCollectionID')"]},"execution_count":14,"metadata":{},"output_type":"execute_result"}],"source":["usa_df[Income_Collections].index.unique()"]},{"cell_type":"markdown","metadata":{},"source":["#### Analysis variables for Data Collections\n","Once we know the data collection we would like to use, we can look at all the unique variables available in that data collection using its unique ID. Let's discover `analysisVariable`s for some of the data collections."]},{"cell_type":"markdown","metadata":{},"source":["__Analysis variables for `Age` data collection__"]},{"cell_type":"code","execution_count":15,"metadata":{},"outputs":[{"data":{"text/plain":["array(['Age.MALE0', 'Age.MALE5', 'Age.MALE10', 'Age.MALE15', 'Age.MALE20',\n","       'Age.MALE25', 'Age.MALE30', 'Age.MALE35', 'Age.MALE40',\n","       'Age.MALE45', 'Age.MALE50', 'Age.MALE55', 'Age.MALE60',\n","       'Age.MALE65', 'Age.MALE70', 'Age.MALE75', 'Age.MALE80',\n","       'Age.MALE85', 'Age.FEM0', 'Age.FEM5', 'Age.FEM10', 'Age.FEM15',\n","       'Age.FEM20', 'Age.FEM25', 'Age.FEM30', 'Age.FEM35', 'Age.FEM40',\n","       'Age.FEM45', 'Age.FEM50', 'Age.FEM55', 'Age.FEM60', 'Age.FEM65',\n","       'Age.FEM70', 'Age.FEM75', 'Age.FEM80', 'Age.FEM85'], dtype=object)"]},"execution_count":15,"metadata":{},"output_type":"execute_result"}],"source":["usa_df.loc['Age']['analysisVariable'].unique()"]},{"cell_type":"markdown","metadata":{},"source":["Analysis variables are typically represented as `dataCollectionID.<analysis variable name>` as seen above."]},{"cell_type":"markdown","metadata":{},"source":["__Analysis variables for `DaytimePopulation` data collection__"]},{"cell_type":"code","execution_count":20,"metadata":{},"outputs":[{"data":{"text/plain":["array(['DaytimePopulation.DPOP_CY', 'DaytimePopulation.DPOPWRK_CY',\n","       'DaytimePopulation.DPOPRES_CY', 'DaytimePopulation.DPOPDENSCY'],\n","      dtype=object)"]},"execution_count":20,"metadata":{},"output_type":"execute_result"}],"source":["usa_df.loc['DaytimePopulation']['analysisVariable'].unique()"]},{"cell_type":"markdown","metadata":{},"source":["### Data Collections for Another Country"]},{"cell_type":"markdown","metadata":{},"source":["Let's look at data collections for New Zealand. [Data Browser](https://doc.arcgis.com/en/esri-demographics/data/data-browser.htm) can be used to examine the entire global listing of variables, and associated datasets for New Zealand.\n","\n","In order to discover the data collections for a particular country, you may first access the reference variable to it using the `country.get()` method, and then fetch the data collections from `country.data_collections` property. Once we know the data collection we would like to use, we can look at `analysisVariable`s available in that data collection."]},{"cell_type":"code","execution_count":21,"metadata":{},"outputs":[{"data":{"text/plain":["arcgis.geoenrichment.enrichment.Country"]},"execution_count":21,"metadata":{},"output_type":"execute_result"}],"source":["# Get US as a country\n","nz = Country.get('New Zealand')\n","type(nz)"]},{"cell_type":"code","execution_count":22,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>analysisVariable</th>\n","      <th>alias</th>\n","      <th>fieldCategory</th>\n","      <th>vintage</th>\n","    </tr>\n","    <tr>\n","      <th>dataCollectionID</th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>15YearIncrements</th>\n","      <td>15YearIncrements.PAGE01_CY</td>\n","      <td>2020 Total Population Age 0-14</td>\n","      <td>2020 Population Totals (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>15YearIncrements</th>\n","      <td>15YearIncrements.PAGE02_CY</td>\n","      <td>2020 Total Population Age 15-29</td>\n","      <td>2020 Population Totals (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>15YearIncrements</th>\n","      <td>15YearIncrements.PAGE03_CY</td>\n","      <td>2020 Total Population Age 30-44</td>\n","      <td>2020 Population Totals (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>15YearIncrements</th>\n","      <td>15YearIncrements.PAGE04_CY</td>\n","      <td>2020 Total Population Age 45-59</td>\n","      <td>2020 Population Totals (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>15YearIncrements</th>\n","      <td>15YearIncrements.PAGE05_CY</td>\n","      <td>2020 Total Population Age 60+</td>\n","      <td>2020 Population Totals (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                            analysisVariable                            alias  \\\n","dataCollectionID                                                                \n","15YearIncrements  15YearIncrements.PAGE01_CY   2020 Total Population Age 0-14   \n","15YearIncrements  15YearIncrements.PAGE02_CY  2020 Total Population Age 15-29   \n","15YearIncrements  15YearIncrements.PAGE03_CY  2020 Total Population Age 30-44   \n","15YearIncrements  15YearIncrements.PAGE04_CY  2020 Total Population Age 45-59   \n","15YearIncrements  15YearIncrements.PAGE05_CY    2020 Total Population Age 60+   \n","\n","                                 fieldCategory vintage  \n","dataCollectionID                                        \n","15YearIncrements  2020 Population Totals (MBR)    2020  \n","15YearIncrements  2020 Population Totals (MBR)    2020  \n","15YearIncrements  2020 Population Totals (MBR)    2020  \n","15YearIncrements  2020 Population Totals (MBR)    2020  \n","15YearIncrements  2020 Population Totals (MBR)    2020  "]},"execution_count":22,"metadata":{},"output_type":"execute_result"}],"source":["nz_df = nz.data_collections\n","\n","# print a few rows of the DataFrame\n","nz_df.head()"]},{"cell_type":"code","execution_count":23,"metadata":{},"outputs":[{"data":{"text/plain":["(193, 4)"]},"execution_count":23,"metadata":{},"output_type":"execute_result"}],"source":["nz_df.shape"]},{"cell_type":"markdown","metadata":{},"source":["#### Unique Data Collections for New Zealand"]},{"cell_type":"markdown","metadata":{},"source":["Let's get a list of unique data collections that are available for New Zealand."]},{"cell_type":"code","execution_count":24,"metadata":{},"outputs":[{"data":{"text/plain":["Index(['15YearIncrements', 'EducationalAttainment', 'Gender',\n","       'HouseholdsbyIncome', 'HouseholdsbyType', 'HouseholdTotals', 'KeyFacts',\n","       'KeyGlobalFacts', 'MaritalStatus', 'PopulationTotals',\n","       'PurchasingPower', 'Spending'],\n","      dtype='object', name='dataCollectionID')"]},"execution_count":24,"metadata":{},"output_type":"execute_result"}],"source":["nz_df.index.unique()"]},{"cell_type":"markdown","metadata":{},"source":["New Zealand has 12 unique data collections.\n","\n","We can look at the `fieldCategory` column to understand each category better."]},{"cell_type":"code","execution_count":25,"metadata":{},"outputs":[{"data":{"text/plain":["array(['2020 Population Totals (MBR)',\n","       '2020 Male Population Totals (MBR)',\n","       '2020 Female Population Totals (MBR)',\n","       '2020 Educational Attainment (MBR)',\n","       '2020 Households by Income (MBR)', '2020 Households by Type (MBR)',\n","       '2020 Household Totals (MBR)', '2020 Marital Status (MBR)',\n","       '2020 Purchasing Power (MBR)', 'Key Demographic Indicators',\n","       'Age: 5 Year Increments',\n","       '2020 Food & Beverage Expenditures (MBR)',\n","       '2020 Alcoholic Beverage Expenditures (MBR)',\n","       '2020 Tobacco Expenditures (MBR)',\n","       '2020 Clothing Expenditures (MBR)',\n","       '2020 Footwear Expenditures (MBR)',\n","       '2020 Furniture & Furnishing Expenditures (MBR)',\n","       '2020 Household Textiles Expenditures (MBR)',\n","       '2020 Household Appliances Expenditures (MBR)',\n","       '2020 Household Utensils Expenditures (MBR)',\n","       '2020 House & Garden Expenditures (MBR)',\n","       '2020 Household Maintenance Expenditures (MBR)',\n","       '2020 Medical Products & Supplies Expenditures (MBR)',\n","       '2020 Consumer Electronics Expenditures (MBR)',\n","       '2020 Recreation & Culture Durable Expenditures (MBR)',\n","       '2020 Entertainment Expenditures (MBR)',\n","       '2020 Recreational & Cultural Service Expenditures (MBR)',\n","       '2020 Books & Stationery Expenditures (MBR)',\n","       '2020 Catering Services Expenditures (MBR)',\n","       '2020 Personal Care Expenditures (MBR)',\n","       '2020 Jewelry & Personal Effects Expenditures (MBR)'], dtype=object)"]},"execution_count":25,"metadata":{},"output_type":"execute_result"}],"source":["nz_df.fieldCategory.unique()"]},{"cell_type":"markdown","metadata":{},"source":["Looking at `fieldCategory` is a great way to clearly understand what the data collection is about. However, to query a data collection its unique ID (`dataCollectionID`) must be used."]},{"cell_type":"markdown","metadata":{},"source":["#### Data Collections for Socio-demographic Factors"]},{"cell_type":"markdown","metadata":{},"source":["New Zealand has fewer `data_collections` compared to U.S. Let's look at data collections for Key Facts, Education and Spending."]},{"cell_type":"markdown","metadata":{},"source":["__Data Collection for Key Facts__"]},{"cell_type":"code","execution_count":26,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>analysisVariable</th>\n","      <th>alias</th>\n","      <th>fieldCategory</th>\n","      <th>vintage</th>\n","    </tr>\n","    <tr>\n","      <th>dataCollectionID</th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>KeyGlobalFacts</th>\n","      <td>KeyGlobalFacts.TOTPOP</td>\n","      <td>Total Population</td>\n","      <td>Key Demographic Indicators</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>KeyGlobalFacts</th>\n","      <td>KeyGlobalFacts.TOTHH</td>\n","      <td>Total Households</td>\n","      <td>Key Demographic Indicators</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>KeyGlobalFacts</th>\n","      <td>KeyGlobalFacts.AVGHHSZ</td>\n","      <td>Average Household Size</td>\n","      <td>Key Demographic Indicators</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>KeyGlobalFacts</th>\n","      <td>KeyGlobalFacts.TOTMALES</td>\n","      <td>Male Population</td>\n","      <td>Age: 5 Year Increments</td>\n","      <td>NaN</td>\n","    </tr>\n","    <tr>\n","      <th>KeyGlobalFacts</th>\n","      <td>KeyGlobalFacts.TOTFEMALES</td>\n","      <td>Female Population</td>\n","      <td>Age: 5 Year Increments</td>\n","      <td>NaN</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                           analysisVariable                   alias  \\\n","dataCollectionID                                                      \n","KeyGlobalFacts        KeyGlobalFacts.TOTPOP        Total Population   \n","KeyGlobalFacts         KeyGlobalFacts.TOTHH        Total Households   \n","KeyGlobalFacts       KeyGlobalFacts.AVGHHSZ  Average Household Size   \n","KeyGlobalFacts      KeyGlobalFacts.TOTMALES         Male Population   \n","KeyGlobalFacts    KeyGlobalFacts.TOTFEMALES       Female Population   \n","\n","                               fieldCategory vintage  \n","dataCollectionID                                      \n","KeyGlobalFacts    Key Demographic Indicators     NaN  \n","KeyGlobalFacts    Key Demographic Indicators     NaN  \n","KeyGlobalFacts    Key Demographic Indicators     NaN  \n","KeyGlobalFacts        Age: 5 Year Increments     NaN  \n","KeyGlobalFacts        Age: 5 Year Increments     NaN  "]},"execution_count":26,"metadata":{},"output_type":"execute_result"}],"source":["nz_df.loc['KeyGlobalFacts']"]},{"cell_type":"markdown","metadata":{},"source":["__Data Collection for Education__"]},{"cell_type":"code","execution_count":27,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>analysisVariable</th>\n","      <th>alias</th>\n","      <th>fieldCategory</th>\n","      <th>vintage</th>\n","    </tr>\n","    <tr>\n","      <th>dataCollectionID</th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC01A_CY</td>\n","      <td>2020 Pop 15+/Edu: No Qualification</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC02A_CY</td>\n","      <td>2020 Pop 15+/Edu: Level 1</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC03A_CY</td>\n","      <td>2020 Pop 15+/Edu: Level 2</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC04A_CY</td>\n","      <td>2020 Pop 15+/Edu: Level 3</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC05A_CY</td>\n","      <td>2020 Pop 15+/Edu: Level 4</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC06B_CY</td>\n","      <td>2020 Pop 15+/Edu: Level 5 Diploma</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC07A_CY</td>\n","      <td>2020 Pop 15+/Edu: Level 6 Diploma</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC08A_CY</td>\n","      <td>2020 Pop 15+/Edu: Bachelor Degree</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC09A_CY</td>\n","      <td>2020 Pop 15+/Edu: Post-graduate and Honours de...</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC10A_CY</td>\n","      <td>2020 Pop 15+/Edu: Master's Degree</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC11A_CY</td>\n","      <td>2020 Pop 15+/Edu: Doctorate</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC12A_CY</td>\n","      <td>2020 Pop 15+/Edu: Overseas Secondary School</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>EducationalAttainment</th>\n","      <td>EducationalAttainment.EDUC13_CY</td>\n","      <td>2020 Pop 15+/Edu: Not Included Elsewhere</td>\n","      <td>2020 Educational Attainment (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                       analysisVariable  \\\n","dataCollectionID                                          \n","EducationalAttainment  EducationalAttainment.EDUC01A_CY   \n","EducationalAttainment  EducationalAttainment.EDUC02A_CY   \n","EducationalAttainment  EducationalAttainment.EDUC03A_CY   \n","EducationalAttainment  EducationalAttainment.EDUC04A_CY   \n","EducationalAttainment  EducationalAttainment.EDUC05A_CY   \n","EducationalAttainment  EducationalAttainment.EDUC06B_CY   \n","EducationalAttainment  EducationalAttainment.EDUC07A_CY   \n","EducationalAttainment  EducationalAttainment.EDUC08A_CY   \n","EducationalAttainment  EducationalAttainment.EDUC09A_CY   \n","EducationalAttainment  EducationalAttainment.EDUC10A_CY   \n","EducationalAttainment  EducationalAttainment.EDUC11A_CY   \n","EducationalAttainment  EducationalAttainment.EDUC12A_CY   \n","EducationalAttainment   EducationalAttainment.EDUC13_CY   \n","\n","                                                                   alias  \\\n","dataCollectionID                                                           \n","EducationalAttainment                 2020 Pop 15+/Edu: No Qualification   \n","EducationalAttainment                          2020 Pop 15+/Edu: Level 1   \n","EducationalAttainment                          2020 Pop 15+/Edu: Level 2   \n","EducationalAttainment                          2020 Pop 15+/Edu: Level 3   \n","EducationalAttainment                          2020 Pop 15+/Edu: Level 4   \n","EducationalAttainment                  2020 Pop 15+/Edu: Level 5 Diploma   \n","EducationalAttainment                  2020 Pop 15+/Edu: Level 6 Diploma   \n","EducationalAttainment                  2020 Pop 15+/Edu: Bachelor Degree   \n","EducationalAttainment  2020 Pop 15+/Edu: Post-graduate and Honours de...   \n","EducationalAttainment                  2020 Pop 15+/Edu: Master's Degree   \n","EducationalAttainment                        2020 Pop 15+/Edu: Doctorate   \n","EducationalAttainment        2020 Pop 15+/Edu: Overseas Secondary School   \n","EducationalAttainment           2020 Pop 15+/Edu: Not Included Elsewhere   \n","\n","                                           fieldCategory vintage  \n","dataCollectionID                                                  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  \n","EducationalAttainment  2020 Educational Attainment (MBR)    2020  "]},"execution_count":27,"metadata":{},"output_type":"execute_result"}],"source":["nz_df.loc['EducationalAttainment']"]},{"cell_type":"markdown","metadata":{},"source":["__Data Collection for Spending__"]},{"cell_type":"code","execution_count":28,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>analysisVariable</th>\n","      <th>alias</th>\n","      <th>fieldCategory</th>\n","      <th>vintage</th>\n","    </tr>\n","    <tr>\n","      <th>dataCollectionID</th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","      <th></th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>Spending</th>\n","      <td>Spending.CS01_CY</td>\n","      <td>2020 Food &amp; Beverage: Total</td>\n","      <td>2020 Food &amp; Beverage Expenditures (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>Spending</th>\n","      <td>Spending.CS01PRM_CY</td>\n","      <td>2020 Food &amp; Beverage: Per Mill</td>\n","      <td>2020 Food &amp; Beverage Expenditures (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>Spending</th>\n","      <td>Spending.CSPC01_CY</td>\n","      <td>2020 Food &amp; Beverage: Per Capita</td>\n","      <td>2020 Food &amp; Beverage Expenditures (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>Spending</th>\n","      <td>Spending.CS01IDX_CY</td>\n","      <td>2020 Food &amp; Beverage: Index</td>\n","      <td>2020 Food &amp; Beverage Expenditures (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>Spending</th>\n","      <td>Spending.CS02_CY</td>\n","      <td>2020 Alcoholic Beverage: Total</td>\n","      <td>2020 Alcoholic Beverage Expenditures (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>Spending</th>\n","      <td>Spending.CS19IDX_CY</td>\n","      <td>2020 Personal Care: Index</td>\n","      <td>2020 Personal Care Expenditures (MBR)</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>Spending</th>\n","      <td>Spending.CS20_CY</td>\n","      <td>2020 Personal Effects: Total</td>\n","      <td>2020 Jewelry &amp; Personal Effects Expenditures (...</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>Spending</th>\n","      <td>Spending.CS20PRM_CY</td>\n","      <td>2020 Personal Effects: Per Mill</td>\n","      <td>2020 Jewelry &amp; Personal Effects Expenditures (...</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>Spending</th>\n","      <td>Spending.CSPC20_CY</td>\n","      <td>2020 Personal Effects: Per Capita</td>\n","      <td>2020 Jewelry &amp; Personal Effects Expenditures (...</td>\n","      <td>2020</td>\n","    </tr>\n","    <tr>\n","      <th>Spending</th>\n","      <td>Spending.CS20IDX_CY</td>\n","      <td>2020 Personal Effects: Index</td>\n","      <td>2020 Jewelry &amp; Personal Effects Expenditures (...</td>\n","      <td>2020</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>80 rows × 4 columns</p>\n","</div>"],"text/plain":["                     analysisVariable                              alias  \\\n","dataCollectionID                                                           \n","Spending             Spending.CS01_CY        2020 Food & Beverage: Total   \n","Spending          Spending.CS01PRM_CY     2020 Food & Beverage: Per Mill   \n","Spending           Spending.CSPC01_CY   2020 Food & Beverage: Per Capita   \n","Spending          Spending.CS01IDX_CY        2020 Food & Beverage: Index   \n","Spending             Spending.CS02_CY     2020 Alcoholic Beverage: Total   \n","...                               ...                                ...   \n","Spending          Spending.CS19IDX_CY          2020 Personal Care: Index   \n","Spending             Spending.CS20_CY       2020 Personal Effects: Total   \n","Spending          Spending.CS20PRM_CY    2020 Personal Effects: Per Mill   \n","Spending           Spending.CSPC20_CY  2020 Personal Effects: Per Capita   \n","Spending          Spending.CS20IDX_CY       2020 Personal Effects: Index   \n","\n","                                                      fieldCategory vintage  \n","dataCollectionID                                                             \n","Spending                    2020 Food & Beverage Expenditures (MBR)    2020  \n","Spending                    2020 Food & Beverage Expenditures (MBR)    2020  \n","Spending                    2020 Food & Beverage Expenditures (MBR)    2020  \n","Spending                    2020 Food & Beverage Expenditures (MBR)    2020  \n","Spending                 2020 Alcoholic Beverage Expenditures (MBR)    2020  \n","...                                                             ...     ...  \n","Spending                      2020 Personal Care Expenditures (MBR)    2020  \n","Spending          2020 Jewelry & Personal Effects Expenditures (...    2020  \n","Spending          2020 Jewelry & Personal Effects Expenditures (...    2020  \n","Spending          2020 Jewelry & Personal Effects Expenditures (...    2020  \n","Spending          2020 Jewelry & Personal Effects Expenditures (...    2020  \n","\n","[80 rows x 4 columns]"]},"execution_count":28,"metadata":{},"output_type":"execute_result"}],"source":["nz_df.loc['Spending']"]},{"cell_type":"markdown","metadata":{},"source":["#### Analysis variables for Data Collections\n","Once we know the data collection we would like to use, we can look at all the unique variables available in that data collection using its unique ID. Let's discover `analysisVariable`s for some of the data collections we looked at earlier."]},{"cell_type":"markdown","metadata":{},"source":["__Analysis variables for `KeyGlobalFacts` data collection__"]},{"cell_type":"code","execution_count":29,"metadata":{},"outputs":[{"data":{"text/plain":["array(['KeyGlobalFacts.TOTPOP', 'KeyGlobalFacts.TOTHH',\n","       'KeyGlobalFacts.AVGHHSZ', 'KeyGlobalFacts.TOTMALES',\n","       'KeyGlobalFacts.TOTFEMALES'], dtype=object)"]},"execution_count":29,"metadata":{},"output_type":"execute_result"}],"source":["nz_df.loc['KeyGlobalFacts']['analysisVariable'].unique()"]},{"cell_type":"markdown","metadata":{},"source":["__Analysis variables for `EducationalAttainment` data collection__"]},{"cell_type":"code","execution_count":30,"metadata":{},"outputs":[{"data":{"text/plain":["array(['EducationalAttainment.EDUC01A_CY',\n","       'EducationalAttainment.EDUC02A_CY',\n","       'EducationalAttainment.EDUC03A_CY',\n","       'EducationalAttainment.EDUC04A_CY',\n","       'EducationalAttainment.EDUC05A_CY',\n","       'EducationalAttainment.EDUC06B_CY',\n","       'EducationalAttainment.EDUC07A_CY',\n","       'EducationalAttainment.EDUC08A_CY',\n","       'EducationalAttainment.EDUC09A_CY',\n","       'EducationalAttainment.EDUC10A_CY',\n","       'EducationalAttainment.EDUC11A_CY',\n","       'EducationalAttainment.EDUC12A_CY',\n","       'EducationalAttainment.EDUC13_CY'], dtype=object)"]},"execution_count":30,"metadata":{},"output_type":"execute_result"}],"source":["nz_df.loc['EducationalAttainment']['analysisVariable'].unique()"]},{"cell_type":"markdown","metadata":{},"source":["__Analysis variables for `Spending` data collection__"]},{"cell_type":"code","execution_count":31,"metadata":{},"outputs":[{"data":{"text/plain":["array(['Spending.CS01_CY', 'Spending.CS01PRM_CY', 'Spending.CSPC01_CY',\n","       'Spending.CS01IDX_CY', 'Spending.CS02_CY', 'Spending.CS02PRM_CY',\n","       'Spending.CSPC02_CY', 'Spending.CS02IDX_CY', 'Spending.CS03_CY',\n","       'Spending.CS03PRM_CY', 'Spending.CSPC03_CY', 'Spending.CS03IDX_CY',\n","       'Spending.CS04_CY', 'Spending.CS04PRM_CY', 'Spending.CSPC04_CY',\n","       'Spending.CS04IDX_CY', 'Spending.CS05_CY', 'Spending.CS05PRM_CY',\n","       'Spending.CSPC05_CY', 'Spending.CS05IDX_CY', 'Spending.CS06_CY',\n","       'Spending.CS06PRM_CY', 'Spending.CSPC06_CY', 'Spending.CS06IDX_CY',\n","       'Spending.CS07_CY', 'Spending.CS07PRM_CY', 'Spending.CSPC07_CY',\n","       'Spending.CS07IDX_CY', 'Spending.CS08_CY', 'Spending.CS08PRM_CY',\n","       'Spending.CSPC08_CY', 'Spending.CS08IDX_CY', 'Spending.CS09_CY',\n","       'Spending.CS09PRM_CY', 'Spending.CSPC09_CY', 'Spending.CS09IDX_CY',\n","       'Spending.CS10_CY', 'Spending.CS10PRM_CY', 'Spending.CSPC10_CY',\n","       'Spending.CS10IDX_CY', 'Spending.CS11_CY', 'Spending.CS11PRM_CY',\n","       'Spending.CSPC11_CY', 'Spending.CS11IDX_CY', 'Spending.CS12_CY',\n","       'Spending.CS12PRM_CY', 'Spending.CSPC12_CY', 'Spending.CS12IDX_CY',\n","       'Spending.CS13_CY', 'Spending.CS13PRM_CY', 'Spending.CSPC13_CY',\n","       'Spending.CS13IDX_CY', 'Spending.CS14_CY', 'Spending.CS14PRM_CY',\n","       'Spending.CSPC14_CY', 'Spending.CS14IDX_CY', 'Spending.CS15_CY',\n","       'Spending.CS15PRM_CY', 'Spending.CSPC15_CY', 'Spending.CS15IDX_CY',\n","       'Spending.CS16_CY', 'Spending.CS16PRM_CY', 'Spending.CSPC16_CY',\n","       'Spending.CS16IDX_CY', 'Spending.CS17_CY', 'Spending.CS17PRM_CY',\n","       'Spending.CSPC17_CY', 'Spending.CS17IDX_CY', 'Spending.CS18_CY',\n","       'Spending.CS18PRM_CY', 'Spending.CSPC18_CY', 'Spending.CS18IDX_CY',\n","       'Spending.CS19_CY', 'Spending.CS19PRM_CY', 'Spending.CSPC19_CY',\n","       'Spending.CS19IDX_CY', 'Spending.CS20_CY', 'Spending.CS20PRM_CY',\n","       'Spending.CSPC20_CY', 'Spending.CS20IDX_CY'], dtype=object)"]},"execution_count":31,"metadata":{},"output_type":"execute_result"}],"source":["nz_df.loc['Spending']['analysisVariable'].unique()"]},{"cell_type":"markdown","metadata":{},"source":["### Perform Enrichment using Data Collections and Analysis Variables"]},{"cell_type":"markdown","metadata":{},"source":["Data Collections can be used to enrich various study areas. `data_collection`s and `analysis_variable`s  can be passed in the `enrich()` method. Details about enriching study areas can be found in __Enriching Study Areas__ section. \n","\n","Let's look at a few similar examples of GeoEnrichment here."]},{"cell_type":"markdown","metadata":{},"source":["#### Enrich using Data Collections"]},{"cell_type":"markdown","metadata":{},"source":["__Enrich with `Age` data collection__\n","\n","Here we see an address being enriched by data from `Age` data collection."]},{"cell_type":"code","execution_count":32,"metadata":{},"outputs":[],"source":["# Enriching single address as single line imput\n","age_coll = enrich(study_areas=[\"380 New York St Redlands CA 92373\"], \n","                       data_collections=['Age'])"]},{"cell_type":"code","execution_count":33,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>source_country</th>\n","      <th>x</th>\n","      <th>y</th>\n","      <th>area_type</th>\n","      <th>buffer_units</th>\n","      <th>buffer_units_alias</th>\n","      <th>buffer_radii</th>\n","      <th>aggregation_method</th>\n","      <th>population_to_polygon_size_rating</th>\n","      <th>apportionment_confidence</th>\n","      <th>...</th>\n","      <th>fem45</th>\n","      <th>fem50</th>\n","      <th>fem55</th>\n","      <th>fem60</th>\n","      <th>fem65</th>\n","      <th>fem70</th>\n","      <th>fem75</th>\n","      <th>fem80</th>\n","      <th>fem85</th>\n","      <th>SHAPE</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>USA</td>\n","      <td>-117.19479</td>\n","      <td>34.057265</td>\n","      <td>RingBuffer</td>\n","      <td>esriMiles</td>\n","      <td>Miles</td>\n","      <td>1.0</td>\n","      <td>BlockApportionment:US.BlockGroups;PointsLayer:...</td>\n","      <td>2.191</td>\n","      <td>2.576</td>\n","      <td>...</td>\n","      <td>366.0</td>\n","      <td>392.0</td>\n","      <td>365.0</td>\n","      <td>345.0</td>\n","      <td>322.0</td>\n","      <td>277.0</td>\n","      <td>168.0</td>\n","      <td>103.0</td>\n","      <td>132.0</td>\n","      <td>{\"rings\": [[[-117.19479001927878, 34.071773611...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>1 rows × 48 columns</p>\n","</div>"],"text/plain":["  source_country          x          y   area_type buffer_units  \\\n","0            USA -117.19479  34.057265  RingBuffer    esriMiles   \n","\n","  buffer_units_alias  buffer_radii  \\\n","0              Miles           1.0   \n","\n","                                  aggregation_method  \\\n","0  BlockApportionment:US.BlockGroups;PointsLayer:...   \n","\n","   population_to_polygon_size_rating  apportionment_confidence  ...  fem45  \\\n","0                              2.191                     2.576  ...  366.0   \n","\n","   fem50  fem55  fem60  fem65  fem70  fem75  fem80  fem85  \\\n","0  392.0  365.0  345.0  322.0  277.0  168.0  103.0  132.0   \n","\n","                                               SHAPE  \n","0  {\"rings\": [[[-117.19479001927878, 34.071773611...  \n","\n","[1 rows x 48 columns]"]},"execution_count":33,"metadata":{},"output_type":"execute_result"}],"source":["age_coll"]},{"cell_type":"code","execution_count":34,"metadata":{},"outputs":[{"data":{"text/plain":["Index(['source_country', 'x', 'y', 'area_type', 'buffer_units',\n","       'buffer_units_alias', 'buffer_radii', 'aggregation_method',\n","       'population_to_polygon_size_rating', 'apportionment_confidence',\n","       'has_data', 'male0', 'male5', 'male10', 'male15', 'male20', 'male25',\n","       'male30', 'male35', 'male40', 'male45', 'male50', 'male55', 'male60',\n","       'male65', 'male70', 'male75', 'male80', 'male85', 'fem0', 'fem5',\n","       'fem10', 'fem15', 'fem20', 'fem25', 'fem30', 'fem35', 'fem40', 'fem45',\n","       'fem50', 'fem55', 'fem60', 'fem65', 'fem70', 'fem75', 'fem80', 'fem85',\n","       'SHAPE'],\n","      dtype='object')"]},"execution_count":34,"metadata":{},"output_type":"execute_result"}],"source":["age_coll.columns"]},{"cell_type":"markdown","metadata":{},"source":["When a data collection is specified without specific analysis variables, all variables under the data collection are used for enrichment as can be seen above."]},{"cell_type":"markdown","metadata":{},"source":["__Enrich with `Health` data collection__\n","\n","Here we see a zip code being enriched by data from Health data collection."]},{"cell_type":"code","execution_count":35,"metadata":{},"outputs":[],"source":["redlands = usa.subgeographies.states['California'].zip5['92373']"]},{"cell_type":"code","execution_count":36,"metadata":{},"outputs":[],"source":["redlands_df = enrich(study_areas=[redlands], data_collections=['Health'] )"]},{"cell_type":"code","execution_count":37,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>std_geography_level</th>\n","      <th>std_geography_name</th>\n","      <th>std_geography_id</th>\n","      <th>source_country</th>\n","      <th>aggregation_method</th>\n","      <th>population_to_polygon_size_rating</th>\n","      <th>apportionment_confidence</th>\n","      <th>has_data</th>\n","      <th>rel65_hi2_oc</th>\n","      <th>acscivnins</th>\n","      <th>...</th>\n","      <th>pop85_cy</th>\n","      <th>pop18up_cy</th>\n","      <th>pop21up_cy</th>\n","      <th>medage_cy</th>\n","      <th>hhu18_c10</th>\n","      <th>medhinc_cy</th>\n","      <th>s27_bus</th>\n","      <th>s27_sales</th>\n","      <th>s27_emp</th>\n","      <th>SHAPE</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>US.ZIP5</td>\n","      <td>Redlands</td>\n","      <td>92373</td>\n","      <td>USA</td>\n","      <td>Query:US.ZIP5</td>\n","      <td>2.191</td>\n","      <td>2.576</td>\n","      <td>1</td>\n","      <td>2.0</td>\n","      <td>31157.0</td>\n","      <td>...</td>\n","      <td>1205.0</td>\n","      <td>28208.0</td>\n","      <td>27076.0</td>\n","      <td>42.2</td>\n","      <td>3851.0</td>\n","      <td>91009.0</td>\n","      <td>224.0</td>\n","      <td>306371.0</td>\n","      <td>4093.0</td>\n","      <td>{\"rings\": [[[-117.16767396036383, 33.976847519...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>1 rows × 431 columns</p>\n","</div>"],"text/plain":["  std_geography_level std_geography_name std_geography_id source_country  \\\n","0             US.ZIP5           Redlands            92373            USA   \n","\n","  aggregation_method  population_to_polygon_size_rating  \\\n","0      Query:US.ZIP5                              2.191   \n","\n","   apportionment_confidence  has_data  rel65_hi2_oc  acscivnins  ...  \\\n","0                     2.576         1           2.0     31157.0  ...   \n","\n","   pop85_cy  pop18up_cy  pop21up_cy  medage_cy  hhu18_c10  medhinc_cy  \\\n","0    1205.0     28208.0     27076.0       42.2     3851.0     91009.0   \n","\n","   s27_bus  s27_sales  s27_emp  \\\n","0    224.0   306371.0   4093.0   \n","\n","                                               SHAPE  \n","0  {\"rings\": [[[-117.16767396036383, 33.976847519...  \n","\n","[1 rows x 431 columns]"]},"execution_count":37,"metadata":{},"output_type":"execute_result"}],"source":["redlands_df"]},{"cell_type":"code","execution_count":38,"metadata":{},"outputs":[{"data":{"text/plain":["Index(['std_geography_level', 'std_geography_name', 'std_geography_id',\n","       'source_country', 'aggregation_method',\n","       'population_to_polygon_size_rating', 'apportionment_confidence',\n","       'has_data', 'rel65_hi2_oc', 'acscivnins',\n","       ...\n","       'pop85_cy', 'pop18up_cy', 'pop21up_cy', 'medage_cy', 'hhu18_c10',\n","       'medhinc_cy', 's27_bus', 's27_sales', 's27_emp', 'SHAPE'],\n","      dtype='object', length=431)"]},"execution_count":38,"metadata":{},"output_type":"execute_result"}],"source":["redlands_df.columns"]},{"cell_type":"markdown","metadata":{},"source":["#### Enrich using Analysis Variables\n","\n","Data can be enriched by specifying specific analysis variables of a data collection with which we want to enrich our data. In this example, we will look at `analysis_variables` for Age `data_collection` and then use specific analysis variables to `enrich()` a study area."]},{"cell_type":"code","execution_count":39,"metadata":{},"outputs":[{"data":{"text/plain":["array(['Age.MALE0', 'Age.MALE5', 'Age.MALE10', 'Age.MALE15', 'Age.MALE20',\n","       'Age.MALE25', 'Age.MALE30', 'Age.MALE35', 'Age.MALE40',\n","       'Age.MALE45', 'Age.MALE50', 'Age.MALE55', 'Age.MALE60',\n","       'Age.MALE65', 'Age.MALE70', 'Age.MALE75', 'Age.MALE80',\n","       'Age.MALE85', 'Age.FEM0', 'Age.FEM5', 'Age.FEM10', 'Age.FEM15',\n","       'Age.FEM20', 'Age.FEM25', 'Age.FEM30', 'Age.FEM35', 'Age.FEM40',\n","       'Age.FEM45', 'Age.FEM50', 'Age.FEM55', 'Age.FEM60', 'Age.FEM65',\n","       'Age.FEM70', 'Age.FEM75', 'Age.FEM80', 'Age.FEM85'], dtype=object)"]},"execution_count":39,"metadata":{},"output_type":"execute_result"}],"source":["# Unique analysis variables for Age data collection\n","usa = Country.get('US')\n","usa.data_collections.loc['Age']['analysisVariable'].unique()"]},{"cell_type":"markdown","metadata":{},"source":["Now, we will enrich our study area with `Age.FEM45, Age.FEM55, Age.FEM65` variables"]},{"cell_type":"code","execution_count":40,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>source_country</th>\n","      <th>x</th>\n","      <th>y</th>\n","      <th>area_type</th>\n","      <th>buffer_units</th>\n","      <th>buffer_units_alias</th>\n","      <th>buffer_radii</th>\n","      <th>aggregation_method</th>\n","      <th>population_to_polygon_size_rating</th>\n","      <th>apportionment_confidence</th>\n","      <th>has_data</th>\n","      <th>fem45</th>\n","      <th>fem55</th>\n","      <th>fem65</th>\n","      <th>SHAPE</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>USA</td>\n","      <td>-117.19479</td>\n","      <td>34.057265</td>\n","      <td>RingBuffer</td>\n","      <td>esriMiles</td>\n","      <td>Miles</td>\n","      <td>1.0</td>\n","      <td>BlockApportionment:US.BlockGroups;PointsLayer:...</td>\n","      <td>2.191</td>\n","      <td>2.576</td>\n","      <td>1</td>\n","      <td>366.0</td>\n","      <td>365.0</td>\n","      <td>322.0</td>\n","      <td>{\"rings\": [[[-117.19479001927878, 34.071773611...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["  source_country          x          y   area_type buffer_units  \\\n","0            USA -117.19479  34.057265  RingBuffer    esriMiles   \n","\n","  buffer_units_alias  buffer_radii  \\\n","0              Miles           1.0   \n","\n","                                  aggregation_method  \\\n","0  BlockApportionment:US.BlockGroups;PointsLayer:...   \n","\n","   population_to_polygon_size_rating  apportionment_confidence  has_data  \\\n","0                              2.191                     2.576         1   \n","\n","   fem45  fem55  fem65                                              SHAPE  \n","0  366.0  365.0  322.0  {\"rings\": [[[-117.19479001927878, 34.071773611...  "]},"execution_count":40,"metadata":{},"output_type":"execute_result"}],"source":["enrich(study_areas=[\"380 New York St Redlands CA 92373\"], \n","       analysis_variables=[\"Age.FEM45\",\"Age.FEM55\",\"Age.FEM65\"])"]},{"cell_type":"markdown","metadata":{},"source":["## Enriching Spatially Enabled Dataframes"]},{"cell_type":"markdown","metadata":{},"source":["One of the most common use case for GeoEnrichment is enriching existing data in feature layers. As a user, you may need to analyze and enrich your data that already exists in feature layers. Spatially Enabled DataFrame (SeDF) helps us bring the data from layer into a dataframe which can then be GeoEnriched. \n","\n","Let's look at an example using an existing layer of Covid-19 dataset. This feature layer includes latest Covid-19 Cases, Recovered and Deaths data for U.S. at the county level."]},{"cell_type":"code","execution_count":41,"metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["<Item title:\"COVID-19 Cases US\" type:Feature Layer Collection owner:CSSE_covid19>\n"]},{"data":{"text/plain":["<FeatureLayer url:\"https://services1.arcgis.com/0MSEUqKaxRlEPj5g/arcgis/rest/services/ncov_cases_US/FeatureServer/0\">"]},"execution_count":41,"metadata":{},"output_type":"execute_result"}],"source":["# Get the layer\n","gis = GIS(set_active=False)\n","covid_item = gis.content.get('628578697fb24d8ea4c32fa0c5ae1843')\n","print(covid_item)\n","covid_layer = covid_item.layers[0]\n","covid_layer"]},{"cell_type":"markdown","metadata":{},"source":["We can query the layer as a dataframe and then use the dataframe for enrichment."]},{"cell_type":"code","execution_count":42,"metadata":{},"outputs":[{"data":{"text/plain":["(3272, 19)"]},"execution_count":42,"metadata":{},"output_type":"execute_result"}],"source":["covid_df = covid_layer.query(as_df=True)\n","covid_df.shape"]},{"cell_type":"code","execution_count":43,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>OBJECTID</th>\n","      <th>Province_State</th>\n","      <th>Country_Region</th>\n","      <th>Last_Update</th>\n","      <th>Lat</th>\n","      <th>Long_</th>\n","      <th>Confirmed</th>\n","      <th>Recovered</th>\n","      <th>Deaths</th>\n","      <th>Active</th>\n","      <th>Admin2</th>\n","      <th>FIPS</th>\n","      <th>Combined_Key</th>\n","      <th>Incident_Rate</th>\n","      <th>People_Tested</th>\n","      <th>People_Hospitalized</th>\n","      <th>UID</th>\n","      <th>ISO3</th>\n","      <th>SHAPE</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>1</td>\n","      <td>Alabama</td>\n","      <td>US</td>\n","      <td>2022-10-27 17:22:34</td>\n","      <td>32.539527</td>\n","      <td>-86.644082</td>\n","      <td>18511</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>228</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>Autauga</td>\n","      <td>01001</td>\n","      <td>Autauga, Alabama, US</td>\n","      <td>33132.864379</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>84001001</td>\n","      <td>USA</td>\n","      <td>{\"x\": -86.64408226999996, \"y\": 32.539527450000...</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>2</td>\n","      <td>Alabama</td>\n","      <td>US</td>\n","      <td>2022-10-27 17:22:34</td>\n","      <td>30.72775</td>\n","      <td>-87.722071</td>\n","      <td>65973</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>716</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>Baldwin</td>\n","      <td>01003</td>\n","      <td>Baldwin, Alabama, US</td>\n","      <td>29553.293853</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>84001003</td>\n","      <td>USA</td>\n","      <td>{\"x\": -87.72207057999998, \"y\": 30.727749910000...</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>3</td>\n","      <td>Alabama</td>\n","      <td>US</td>\n","      <td>2022-10-27 17:22:34</td>\n","      <td>31.868263</td>\n","      <td>-85.387129</td>\n","      <td>6930</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>103</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>Barbour</td>\n","      <td>01005</td>\n","      <td>Barbour, Alabama, US</td>\n","      <td>28072.591752</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>84001005</td>\n","      <td>USA</td>\n","      <td>{\"x\": -85.38712859999998, \"y\": 31.868263000000...</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>4</td>\n","      <td>Alabama</td>\n","      <td>US</td>\n","      <td>2022-10-27 17:22:34</td>\n","      <td>32.996421</td>\n","      <td>-87.125115</td>\n","      <td>7575</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>108</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>Bibb</td>\n","      <td>01007</td>\n","      <td>Bibb, Alabama, US</td>\n","      <td>33826.024828</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>84001007</td>\n","      <td>USA</td>\n","      <td>{\"x\": -87.12511459999996, \"y\": 32.996420640000...</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>5</td>\n","      <td>Alabama</td>\n","      <td>US</td>\n","      <td>2022-10-27 17:22:34</td>\n","      <td>33.982109</td>\n","      <td>-86.567906</td>\n","      <td>17320</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>258</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>Blount</td>\n","      <td>01009</td>\n","      <td>Blount, Alabama, US</td>\n","      <td>29951.92474</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>&lt;NA&gt;</td>\n","      <td>84001009</td>\n","      <td>USA</td>\n","      <td>{\"x\": -86.56790592999994, \"y\": 33.982109180000...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["   OBJECTID Province_State Country_Region         Last_Update        Lat  \\\n","0         1        Alabama             US 2022-10-27 17:22:34  32.539527   \n","1         2        Alabama             US 2022-10-27 17:22:34   30.72775   \n","2         3        Alabama             US 2022-10-27 17:22:34  31.868263   \n","3         4        Alabama             US 2022-10-27 17:22:34  32.996421   \n","4         5        Alabama             US 2022-10-27 17:22:34  33.982109   \n","\n","       Long_  Confirmed  Recovered  Deaths  Active   Admin2   FIPS  \\\n","0 -86.644082      18511       <NA>     228    <NA>  Autauga  01001   \n","1 -87.722071      65973       <NA>     716    <NA>  Baldwin  01003   \n","2 -85.387129       6930       <NA>     103    <NA>  Barbour  01005   \n","3 -87.125115       7575       <NA>     108    <NA>     Bibb  01007   \n","4 -86.567906      17320       <NA>     258    <NA>   Blount  01009   \n","\n","           Combined_Key  Incident_Rate  People_Tested  People_Hospitalized  \\\n","0  Autauga, Alabama, US   33132.864379           <NA>                 <NA>   \n","1  Baldwin, Alabama, US   29553.293853           <NA>                 <NA>   \n","2  Barbour, Alabama, US   28072.591752           <NA>                 <NA>   \n","3     Bibb, Alabama, US   33826.024828           <NA>                 <NA>   \n","4   Blount, Alabama, US    29951.92474           <NA>                 <NA>   \n","\n","        UID ISO3                                              SHAPE  \n","0  84001001  USA  {\"x\": -86.64408226999996, \"y\": 32.539527450000...  \n","1  84001003  USA  {\"x\": -87.72207057999998, \"y\": 30.727749910000...  \n","2  84001005  USA  {\"x\": -85.38712859999998, \"y\": 31.868263000000...  \n","3  84001007  USA  {\"x\": -87.12511459999996, \"y\": 32.996420640000...  \n","4  84001009  USA  {\"x\": -86.56790592999994, \"y\": 33.982109180000...  "]},"execution_count":43,"metadata":{},"output_type":"execute_result"}],"source":["covid_df.head()"]},{"cell_type":"markdown","metadata":{},"source":["To showcase GeoEnrichment, we will create a subset of the original data and then `enrich()` the subset."]},{"cell_type":"code","execution_count":44,"metadata":{},"outputs":[{"data":{"text/plain":["(100, 19)"]},"execution_count":44,"metadata":{},"output_type":"execute_result"}],"source":["# Create subset\n","test_df = covid_df.iloc[:100].copy()\n","test_df.shape"]},{"cell_type":"code","execution_count":45,"metadata":{},"outputs":[{"data":{"text/plain":["['point', None]"]},"execution_count":45,"metadata":{},"output_type":"execute_result"}],"source":["# Check geometry\n","test_df.spatial.geometry_type"]},{"cell_type":"markdown","metadata":{},"source":["A dataframe can be passed as a value to `study_areas` parameter of the `enrich()` method. Here we are enriching our dataframe with specific variables from `Age` data collection."]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Enrich dataframe\n","new_df = enrich(study_areas=test_df.spatial, \n","       analysis_variables=[\"Age.FEM45\",\"Age.FEM55\",\"Age.FEM65\"])"]},{"cell_type":"code","execution_count":44,"metadata":{},"outputs":[{"data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>ID</th>\n","      <th>OBJECTID_0</th>\n","      <th>sourceCountry</th>\n","      <th>Long_</th>\n","      <th>Recovered</th>\n","      <th>Country_Region</th>\n","      <th>FIPS</th>\n","      <th>Last_Update</th>\n","      <th>Combined_Key</th>\n","      <th>ISO3</th>\n","      <th>...</th>\n","      <th>bufferUnitsAlias</th>\n","      <th>bufferRadii</th>\n","      <th>aggregationMethod</th>\n","      <th>populationToPolygonSizeRating</th>\n","      <th>apportionmentConfidence</th>\n","      <th>HasData</th>\n","      <th>FEM45</th>\n","      <th>FEM55</th>\n","      <th>FEM65</th>\n","      <th>SHAPE</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>US</td>\n","      <td>-82.461707</td>\n","      <td>0</td>\n","      <td>US</td>\n","      <td>45001</td>\n","      <td>1596857725000</td>\n","      <td>Abbeville, South Carolina, US</td>\n","      <td>USA</td>\n","      <td>...</td>\n","      <td>Miles</td>\n","      <td>1</td>\n","      <td>BlockApportionment:US.BlockGroups</td>\n","      <td>2.191</td>\n","      <td>2.576</td>\n","      <td>1</td>\n","      <td>2</td>\n","      <td>2</td>\n","      <td>2</td>\n","      <td>{\"rings\": [[[-82.46170657999994, 34.2378420028...</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>1</td>\n","      <td>2</td>\n","      <td>US</td>\n","      <td>-92.414197</td>\n","      <td>0</td>\n","      <td>US</td>\n","      <td>22001</td>\n","      <td>1596857725000</td>\n","      <td>Acadia, Louisiana, US</td>\n","      <td>USA</td>\n","      <td>...</td>\n","      <td>Miles</td>\n","      <td>1</td>\n","      <td>BlockApportionment:US.BlockGroups</td>\n","      <td>2.191</td>\n","      <td>2.576</td>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>3</td>\n","      <td>2</td>\n","      <td>{\"rings\": [[[-92.41419697999997, 30.3095821536...</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>2</td>\n","      <td>3</td>\n","      <td>US</td>\n","      <td>-75.632346</td>\n","      <td>0</td>\n","      <td>US</td>\n","      <td>51001</td>\n","      <td>1596857725000</td>\n","      <td>Accomack, Virginia, US</td>\n","      <td>USA</td>\n","      <td>...</td>\n","      <td>Miles</td>\n","      <td>1</td>\n","      <td>BlockApportionment:US.BlockGroups</td>\n","      <td>2.191</td>\n","      <td>2.576</td>\n","      <td>1</td>\n","      <td>14</td>\n","      <td>16</td>\n","      <td>14</td>\n","      <td>{\"rings\": [[[-75.63234615, 37.781571251121655]...</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>3</td>\n","      <td>4</td>\n","      <td>US</td>\n","      <td>-116.241552</td>\n","      <td>0</td>\n","      <td>US</td>\n","      <td>16001</td>\n","      <td>1596857725000</td>\n","      <td>Ada, Idaho, US</td>\n","      <td>USA</td>\n","      <td>...</td>\n","      <td>Miles</td>\n","      <td>1</td>\n","      <td>BlockApportionment:US.BlockGroups</td>\n","      <td>2.191</td>\n","      <td>2.576</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>{\"rings\": [[[-116.24155159999998, 43.467142851...</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>4</td>\n","      <td>5</td>\n","      <td>US</td>\n","      <td>-94.471059</td>\n","      <td>0</td>\n","      <td>US</td>\n","      <td>19001</td>\n","      <td>1596857725000</td>\n","      <td>Adair, Iowa, US</td>\n","      <td>USA</td>\n","      <td>...</td>\n","      <td>Miles</td>\n","      <td>1</td>\n","      <td>BlockApportionment:US.BlockGroups</td>\n","      <td>2.191</td>\n","      <td>2.576</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>{\"rings\": [[[-94.47105873999998, 41.3452468224...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>5 rows × 31 columns</p>\n","</div>"],"text/plain":["  ID  OBJECTID_0 sourceCountry       Long_  Recovered Country_Region   FIPS  \\\n","0  0           1            US  -82.461707          0             US  45001   \n","1  1           2            US  -92.414197          0             US  22001   \n","2  2           3            US  -75.632346          0             US  51001   \n","3  3           4            US -116.241552          0             US  16001   \n","4  4           5            US  -94.471059          0             US  19001   \n","\n","     Last_Update                   Combined_Key ISO3  ... bufferUnitsAlias  \\\n","0  1596857725000  Abbeville, South Carolina, US  USA  ...            Miles   \n","1  1596857725000          Acadia, Louisiana, US  USA  ...            Miles   \n","2  1596857725000         Accomack, Virginia, US  USA  ...            Miles   \n","3  1596857725000                 Ada, Idaho, US  USA  ...            Miles   \n","4  1596857725000                Adair, Iowa, US  USA  ...            Miles   \n","\n","   bufferRadii                  aggregationMethod  \\\n","0            1  BlockApportionment:US.BlockGroups   \n","1            1  BlockApportionment:US.BlockGroups   \n","2            1  BlockApportionment:US.BlockGroups   \n","3            1  BlockApportionment:US.BlockGroups   \n","4            1  BlockApportionment:US.BlockGroups   \n","\n","   populationToPolygonSizeRating  apportionmentConfidence  HasData  FEM45  \\\n","0                          2.191                    2.576        1      2   \n","1                          2.191                    2.576        1      3   \n","2                          2.191                    2.576        1     14   \n","3                          2.191                    2.576        1      0   \n","4                          2.191                    2.576        1      1   \n","\n","  FEM55  FEM65                                              SHAPE  \n","0     2      2  {\"rings\": [[[-82.46170657999994, 34.2378420028...  \n","1     3      2  {\"rings\": [[[-92.41419697999997, 30.3095821536...  \n","2    16     14  {\"rings\": [[[-75.63234615, 37.781571251121655]...  \n","3     0      0  {\"rings\": [[[-116.24155159999998, 43.467142851...  \n","4     1      1  {\"rings\": [[[-94.47105873999998, 41.3452468224...  \n","\n","[5 rows x 31 columns]"]},"execution_count":44,"metadata":{},"output_type":"execute_result"}],"source":["new_df.head()"]},{"cell_type":"code","execution_count":45,"metadata":{},"outputs":[],"source":["new_df.drop(['OBJECTID_0', 'ID','Last_Update'], axis=1, inplace=True)"]},{"cell_type":"code","execution_count":52,"metadata":{},"outputs":[{"data":{"text/plain":["(91, 28)"]},"execution_count":52,"metadata":{},"output_type":"execute_result"}],"source":["# Check shape\n","new_df.shape"]},{"cell_type":"markdown","metadata":{},"source":["We can see that enrichment resulted in 91 records and 31 columns. There are some areas in our dataframe for which enrichment information is not available. Hence, we have 91 records instead of 100. Geoenrichment adds some additional columns along with the analysis variables we enriched for and so we see 31 columns however we are dropping duplicates and unnecessary columns to bring the count down to 28 columns."]},{"cell_type":"markdown","metadata":{},"source":["### Visualize on a Map\n","\n","Let's visualize the enriched dataframe on a map. We will use `FEM65` column to classify our data for plotting on the map."]},{"cell_type":"code","execution_count":67,"metadata":{},"outputs":[{"data":{"text/html":["<img src=\"\"/>"],"text/plain":["<IPython.core.display.Image object>"]},"execution_count":67,"metadata":{},"output_type":"execute_result"}],"source":["covid_map = gis.map('USA')\n","covid_map"]},{"cell_type":"code","execution_count":61,"metadata":{},"outputs":[{"data":{"text/plain":["True"]},"execution_count":61,"metadata":{},"output_type":"execute_result"}],"source":["# Plot on a map\n","new_df.spatial.plot(map_widget=covid_map)"]},{"cell_type":"markdown","metadata":{},"source":["## Conclusion"]},{"cell_type":"markdown","metadata":{},"source":["In this part of the `arcgis.geoenrichment` module guide series, you saw how `data_collections` property of a `Country` object lists its available `data_collection`s and `analysis_variable`s. You explored different data collections, their analysis variables and then enriched study areas using the same. Towards the end, you experienced how spatially enabled dataframes can be enriched.\n","\n","In the subsequent pages, you will learn about Generating Reports and Standard Geography Queries."]}],"metadata":{"anaconda-cloud":{},"kernelspec":{"display_name":"arcgispro-dev","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.9.11 [MSC v.1931 64 bit (AMD64)]"},"livereveal":{"scroll":true},"toc":{"base_numbering":1,"nav_menu":{},"number_sections":true,"sideBar":true,"skip_h1_title":true,"title_cell":"Table of Contents","title_sidebar":"Contents","toc_cell":true,"toc_position":{"height":"calc(100% - 180px)","left":"10px","top":"150px","width":"274px"},"toc_section_display":true,"toc_window_display":true},"vscode":{"interpreter":{"hash":"07c13af76457a6f4e5d5b34d0e1bc42b2e017343b05b916c8871f483eed35ce6"}}},"nbformat":4,"nbformat_minor":2}
